Working Paper No.l 


Table of Contents 


P-1 

P-2 

P-3 


Introduction 


Preface 


P-5 

p.14 


The persisting power and costs of the Literacy Myth 

by Harvey J. Graff 

Literacy, Economy and Society by Brian V. Street 
Ending the myth of the 'Literacy Myth' by Stan Jones 


WORKING PAPERS ON LITERACY 

Working Papers on Literacy is an initiative of The Centre for Literacy. Though this occasional series of 
monographs, we hope to broaden perspectives and stimulate debate on literacy-related issues. 

Some of the papers in the series will be reprints of articles that have previously appeared in our 
newsletter, Literacy Across the Curriculum, and have generated interest and requests for copies. Others 
will be papers that we have produced as outcomes of our research projects. 

The views expressed are those of the authors and do not necessarily reflect the philosophy of The 
Centre for Literacy. We welcome responses. 
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PREFACE 

This document is a collection of three parts, two extended papers and one shorter commentary on the 
International Adult Literacy Survey (IALS). These originally appeared in Literacy Across the Curriculum in 
response to an invitation for comment to several prominent international researchers. 

The first of these entitled "The Persisting Power and Costs of the Literacy Myth" by Harvey J. Graff 
appeared in Volume 12, No. 2, Summer 1996. The second "Literacy, Economy and Society — A Review" by 
Brian V. Street was published in Volume 12, No. 3, Fall 1996. The third, "Ending the Myth of the 'Literacy 
Myth" by Stan Jones was published in Volume 12, No.4, Winter 1997. 

These articles have been collected because they reflect a continuing debate on literacy between two 
schools of thought which have been variously called the autonomous and the socio-cultural. 

The International Adult Literacy Survey (IALS) represents a collaborative effort of seven governments in 
the industrialized world to measure the literacy skills of their adult populations. The results have been 
widely disseminated and are being studied as the basis for social economic and educational policy in the 
participating countries. 

Because of the potentially broad implications of the survey, The Centre for Literacy invited 
commentaries from researchers with different disciplinary perspectives and different understandings of 
the nature and purposes of literacy/ies. At the time of printing this collection, the debate among them 
was continuing. 


About the authors 

Harvey J. Graff is a professor of history at the University of Texas at Dallas who has studied and written 
about literacy for more than 20 years. Among his books are The Labyrinth of Literacy: Reflections on 
Literacy, Past and Present (1989), expanded and revised (1995), The Legacies of Literacy (1991) and The 
Literacy Myth (1991) and others, including studies of literacy in the 19th century Canada. 
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The persisting power and costs of the Literacy Myth 

A Comment on Literacy, Economy and Society: Results of the First International Adult Literacy Survey 
(IALS). Organisation for Economic Co-Operation and Development and Statistics Canada. (Paris and 
Ottawa, 1995). 

by Harvey J. Graff, University of Texas at Dallas 


Themes in IALS 

Literacy, Economy, and Society is a landmark achievement with respect to international cooperation and 
comparison (albeit within a narrow band of nations); focus on adults; technical virtuosity of 
measurement, data-gathering and processing, and presentation; and tone of scientific certainty. About 
that, there is no doubt. 

The report unmistakably marks and is marked by the historical moment in traditions in social 
developmental research and capabilities of social science, by the presumptive place of literacy and 
education within them, and by the current fin-de-siecle sense of threat and malaise, East and West. 

In its first paragraph, the report intones the post-World War II developmental consensus: "In recent 
years, adult literacy has come to be seen as crucial to the economic performance of industrialized 
nations." Reflecting a shift in awareness of recent decades, it continues: "Literacy, is no longer defined 
merely in terms of a basic threshold of reading ability, mastered by all those growing up in developed 
countries. Rather, literacy, is now seen as how adults use written information to function in society. 

The fears engendered by the press of the times appear next: "Today, adults need a higher level of 
literacy to function well: society has become more complex and low-skill jobs are disappearing. 
Therefore, inadequate levels of literacy among a broad section of the population potentially threaten 
the strength of economics and the social cohesion of nations." Not surprisingly, the report states that 
these are "high stakes." The study clearly acknowledges its dependence on Irwin Kirsch's U.S. 
Educational Testing Service 1993 Adult Literacy in America. However, whereas Kirsch's study attracted 
great media attention even outside the U.S., the new report, interestingly, has sparked relatively little in 
North America. [This comment was written before release of Reading the Future, The Canada Report on 
IALS. ed] 


Format of IALS 

Literacy. Economy and Society, is a handsome volume. It presents its information attractively. That 
tabular and graphic modes of expression dominate over the far fewer words, however, is unremarked 
(although arithmetic ability is included in the study). This presentation raises implicitly the possibility 
that newer literacies linked to the numerical, graphical and visual exceed the power of traditional 
alphabetic literacy. Does this mark a sign of late twentieth-century literacy that is sometimes 
misconstrued as a "decline in (print) literacy?" Or does it more directly contradict the design, measures, 
and conclusions of the international study with respect to literacy abilities and needs today and 
tomorrow? 
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Problems with IALS 

These questions only begin to suggest the problematic character of Literacy, Economy and Society. More 
than two decades of critical, empirical, and theoretical work - a significant amount of which focuses on 
or includes literacy - is neither acknowledged nor allowed to influence the study's design, conduct or 
reported conclusions. 

This distance, indeed segregation, of the study front significant, sometimes seminal, critical research and 
commentary is more than revealing and unfortunate, it limits the questions asked as well as the 
underlying and overarching assumptions about the place and power of literacy by itself. It also limits the 
interpretation of the empirical indicators and the fuller interpretive web of meanings in which the 
conclusions may be located and made to speak. 

The important historical dimension - my own area of expertise - is ignored (if sometimes wrongly 
permitted to creep into notions of present and future), as are path-breaking recent studies, many of 
them ethnographic, on the actual contexts in which literacy is valued, acquired, practiced, used, abused 
or neglected. Recent research, ranging from the historical like my own to Shirley Brice Heath's or Brian 
Street's ethnographic studies, among others, qualifies notions of literacy as skill, in part through their 
Awareness of multiple literacies; of great inequalities in opportunities to gain, practice and improve, and 
use those literacies; and of the simultaneous difficulty, complexity and limits of literacy. 

Quantitative and comparative research can certainly, be employed for contextual study and 
interpretation, but in more flexible design and interpretive modes than Literacy, Economy and Society 
employs. These measures neither can nor need bear all the investigative and interpretive weight their 
research tradition places upon them. 

Economists now see literacy in more complex connections and contexts too. Literacy alone seldom 
correlates accurately or meaningfully with job attainment and performance (whereas it links more 
directly to social origins and opportunities). A great many jobs in rapidly expanding sectors of service- 
driven economies require little Literacy. 

Traditions of taking literacy by itself - including the narrow and sometimes naive economism the study 
reflects - no longer merit our attention. In overvaluing literacy taken out of its actual contexts, both 
literacy and lives linked to it are diminished unnecessarily, as Mike Rose in particular shows. 

The research cited and the great revision in thinking about literacy challenge, qualify and contradict the 
science and certainty of Literacy, Economy, and Society from the assumptions of its first paragraph to its 
last. Ironically, or perhaps not, the results of (IALS) needlessly circumscribe themselves as they reveal, to 
borrow my own phrase, the persisting power and costs of the literacy myth. 
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Literacy, Economy and Society: A Review 

by Brian V. Street, King's College, University of London 

Contextualizing literacy 

In 1981 Scribner and Cole in The Psychology of literacy published a definitive account of what can 
legitimately be claimed about the relationship between literacy and cognitive processes. After a decade's 
work among the Vai peoples of Liberia, with a team of psychologists and anthropologists, employing a 
battery of tests and observational methods about the uses of literacy in different languages and 
contexts, they came to the conclusion that 'specific practices promote specific skills.' 

There is nothing special about literacy in general as regards its consequences for cognitive skills - in each 
case particular skills and abilities are associated with a particular literacy. 

In the Vai case there were three languages, Vai, Arabic and English, and literacy was variously used for 
letter writing, for religious purposes and for trading and schooling. Particular skills are associated with 
each set of literacy practices. 

The significance of this study and its uses in academic and applied areas since 1980, has been that it 
challenged the dominant belief until then that literacy in itself led to cognitive enhancement and that 
literacy could be described as a generic skill. It is now orthodoxy that literacy practices vary from one 
context to another. 

The IALS survey of 1995 in one sense develops this tradition by extending the range of literacy practices 
from variations within a single cultural group, as in the Vai case, to variation across nation states, in this 
case across seven countries. Armed with a battery of tests and questions, a series of research teams 
entered the homes of adults in 7 countries in Europe and N. America to attempt to establish levels and 
skills of literacy and their variation. Its findings at first glance also appear to reinforce Scribner & Cole's 
seminal insights: the variation to be found across nations is a product of the variation in uses of literacy 
in those contexts - 'specific practices promote specific skills.' Where variation was identified among 
these populations, it could be put down to different social uses and practices of literacy. 

That Polish respondents did not do as well on the scales for 'document literacy and prose literacy' as 
respondents from the other countries may be explained with reference to 'the changing economic 
situation there, as these scales represent a type of literacy likely to be more common in a fully market- 
oriented society' (p. 57). Similarly the higher level tasks create difficulty for many respondents since 'the 
more difficult tasks may also require them to draw on less familiar or more specialised types of 
knowledge beyond that given in the text' (p. 52): those practised at these skills will do better at them on 
the tests. Again, those who perform mainly at the lowest levels in the scales do so because they have 
less 'practice' at the literacy skills involved in these higher levels. 


The Centre for Literacy 


5 



Working Paper No.l 


The people at lower levels, however, report on themselves as having sufficient literacy skills for their 
purposes - they do not rate themselves as 'poor' and indeed many rate themselves as 'excellent' (p. 

109). This suggests that they apprehend Scribner and Cole's insight that it would be inappropriate to 
apply the literacy standards of one domain or cultural to those required in another: literacy is specific to 
context. 

The researchers themselves certainly pay lip service to this position and have done a service in moving 
beyond the dominant view in agency and government surveys that there is a single standard to be 
applied to all, and that there are just two dimensions in measurement - literacy and illiteracy. They 
rightly reject the latter term as unhelpful in contemporary society, with its unfortunate connotations of 
ignorance and deficit. In practice, levels and demands on literacy are varied and are changing all the 
time, so the researchers instead are attempting to design scales that will capture this complexity and 
variation. Instead of a simple literacy/illiteracy dichotomy, they measure literacy in three 'domains' - 
prose literacy, document literacy and quantitative literacy - across which individuals will vary according 
to experience and context. 

However, there are indications throughout the report that the authors, and certainly those who make 
use of the findings, will in practice resist this call to complexity, and continue to privilege certain kinds of 
literacy and certain types of knowledge as superior to others. Indeed, that in a sense is the motivation 
for producing the report in the first place: to tell governments and agencies where their populations are 
failing so that they can put it right. The language of deficit runs through the report of findings and the 
commentaries on it. 

There is an inevitable contradiction between, on the one hand, recognizing the cultural and contextual 
nature of different 'multifaceted' literacy practices, and on the other, representing particular kinds of 
literacy, those of the researchers' own culture - as preferable. This dilemma is a constant theme in the 
qualitative research literature but it is not addressed directly in this report and its absence leaves 
governments, agencies and educators free to represent the report's findings in traditional deficit terms - 
as proof of 'low standards', the need to return to 'basics', and the need for 'remediation' among those 
who, for instance, fail to subscribe to the literacy of 'a fully market-oriented society' and are therefore 
backward and inferior. 

The ways in which the report lends credence to such ethnocentrism, despite the overt efforts of the 
researchers to avoid it and to address complexity, can be found at a number of levels. These include the 
terminology used to refer to variation; the unselfconscious use of tasks that are culturally specific as 
though they could indicate general literacy levels and skills, and the lack of attention to the theoretical 
and methodological debates raised in the vast qualitative research literature on literacy across cultures. 
Although the authors argue that 'the aim is not to establish a single international literacy standard,' they 
lay their work open to being used in that way partly through their own unconscious use of specific 
concepts and standards with reference to literacy. From the outset the context for literacy is set as 
being the economy and political standards of 'developed' and 'market-oriented' societies: 

"Today, adults need a higher level of literacy to function well: society has become more complex and 
low-skill jobs are disappearing. Therefore, inadequate levels of literacy, among a broad section of the 
population potentially threaten the strength of economics and the social cohesion of nations." 
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There is, of course, a considerable literature on the nature of the 'new work order' and its relation to 
literacy practices, and that literature throws into question many of the assumptions in the report, such 
as the disappearance of low level jobs and the kinds of literacy required for new post-Fordist 
employment. 

But even before we address these issues, it is significant that the report has reduced the discussion of 
literacy to these work-oriented issues and to a traditional 'functional' approach to literacy. 

The authors' definition of literacy itself reproduces the standard functional view of early UNESCO 
documents: 'Using printed and written information to function in society: to achieve one's goals, and to 
develop one's knowledge and potential' (p. 14). 

This view of literacy has been elaborated, refined and frequently rejected by more sophisticated recent 
research. But none of this is mentioned in the report, which proceeds as though the definitions and the 
terms are unproblematic and universally agreed. Similarly the language of deficit already hinted at in the 
opening reference to 'inadequate levels of literacy' recurs throughout as a leitmotif with constant 
suggestions of a literacy 'problem,' of 'low' levels, of 'success' or 'failure' in achieving on the tests and of 
'remediation' in putting it right. 

Literacy practices 

The terminology for describing the complexity of literacy practices likewise suggests a reductionist view 
in which what 'really' counts is skills and levels rather than the broader and more complex uses and 
meanings of literacy indicated by such terms as 'practices.' While the term 'literacy practices' is 
frequently employed, there is equally often a slide towards more narrow functionally defined evaluative 
terms; skills, activities, levels, tasks, and abilities are used as though they all meant the same as, and 
were a gloss on, 'literacy practices.' Under the heading of 'literacy practices at work,' for instance, a gloss 
is provided that immediately reduces practices to the test situation: 'most adults must face some 
literacy tasks at work'; and again under 'literacy practices in the community' we are told 'everyone, 
whether employed or not, can engage in literacy activities... respondents were asked to report on their 
everyday reading and writing and to judge how well their abilities served them' (p. 87). 

The shift in one sentence from the broader 'practices, to 'activities' (a more behavourist concept), to 
'tasks' (the term employed for testing items) and then 'abilities' (a cognitive term supposedly, inferred 
from performance on 'tasks'), is symptomatic of the lack of attention to the conceptual apparatus. 

To those working in the qualitative field, in which there is now a large and elaborate research literature, 
the phrase 'literacy practices' has a specific meaning that precisely separates it from the various terms 
noted above. In that literature, 'literacy practices' in the plural indicates recognition that there are 
multiple uses and meanings of literacy in social practice - that literacy is a social practice that cannot be 
reduced simply to skills, abilities or measurable items on a task list; that literacy practices vary from one 
context to another; that people themselves hold varying meanings, often contested, that may not 
coincide with those of academic researchers; and that the uses of meanings of literacy are always 
imbued with power relations. Both the practice and the social relations around literacy are ways in 
which power is exercised and realized. 

While recognizing that the IALS report belongs to a different genre and has its own rigour within that 
research tradition, it is disappointing that the authors make no reference at all to this significant area of 
work, especially as they themselves make constant reference to some of the tenets, such as the 
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'complexity' of literacy, to its cultural character and to the need to get beyond crude and reductionist 
definitions. Even the chapter of the report entitled 'The practice(s) of literacy' has only one reference at 
the end, to Irwin Kirsch's Adult Literacy in America (1993). One is lead to the conclusion that, whatever 
they state about literacy practices and their complexity, in fact the authors still see literacy as simply a 
measurable skill in which those who practice it in a fully market-oriented society' are superior. 

The complex everyday literacies in which people engage religious literacies, community literacies, 
literacies among urban youth, literacy practices involving secrecy, ethnic identity, the construction of 
gender - are all marginalized by this emphasis on one particular form of literacy and the kinds of 
knowledge associated with it. 

Cultural bias & test items 

What is currently available through the 1995 publication provides admittedly a limited account of its 
research methodology and there are no doubt fuller versions of what the researchers actually did and 
why; but the examples given nevertheless indicate the kinds of problems that qualitative researchers 
are likely to raise with this approach. 

Researchers for the IALS project entered the homes of large numbers of adults in the seven countries 
and administered a series of tests and questionnaires. We are not given much evidence for the 
situations themselves, the nature of the encounters, the previous indications through texts or oral 
accounts that had been given to the subjects or the social interactions involved. Apart from the 
'background questionnaires' and the 'series of questions about literacy practices' noted above, the 
researchers also presented respondents with a series of test booklets, taking up to 45 minutes to 
complete. 

The test items used were taken from documents available in all of the countries and usually consisted of 
a piece of text or a diagram or table, about which the respondent was asked questions. Unfortunately 
we are not often given the actual wording of the questions, which are mostly presented in reported 
speech, nor the exact context in which they were asked - crucial methodological issues in any 
sociolinguistic research on interviewing in general and testing in particular. 

What exactly did the respondents believe they were meant to be doing and how committed were they 
to completing such lengthy and unfamiliar tasks? 

The researchers do recognize the possibility of cultural bias in both the content of the tests and the 
setting but claim to have overcome this: 'one way of trying to guard against cultural bias in the results 
was by constructing a large number of tasks - considerably more than would have been needed to 
obtain statistically valid estimates of each person's literacy level' (p.18). 

A quantitative answer is given to what is essentially a qualitative problem: 'culture' by definition is an 
issue of interpretation and frequently of contestation, not simply a matter of statistical validity. 

The test items themselves raise similar problems that those working in the field of language, discourse 
and culture have certainly not resolved but on which they have developed complex theoretical and 
methodological debates and suggestions for further research. Again none of this is referred to, and 
again that might not matter if the authors were not claiming to be offering findings that have cultural 
and linguistic implications. 

An analysis of the discourse of the test items, for instance, suggests that they generally belong to a 
particular form of language that is school and indeed test oriented. 
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These are not the literacy practices of either work or community as these vary across Europe and North 
America; they are those of the academic researchers who thereby privilege particular discourses and 
modes of knowing, particular 'academic literacies.' 

Many of the 'factual' passages are written in a discourse style typical of what Kress has described as pre- 
war school based 'scientific writing,' in contrast with recent genres that employ more oral-like features 
[see BOX 21 . 

The text assumes a very particular kind of audience and no doubt many readers will never have 
encountered such a style, or if so, perhaps only at school which for some would be many years ago. 

This is also evident in the classic testing discourse that assumes the testee knows what is indicated by 
terms like 'suggest' i.e. that they are expected to recognize a question about cause and effect, whereas 
the word may 'suggest' quite different things to them. They are also expected to make the link between 
the word 'suggest' in the question and the word 'indicate' in the text and to differentiate the latter from 
'because' in the previous (distracter) sentence. It is clear that a very particular discourse style is involved 
here, both in the target text and in the question asked about it. Recent works on the language of the 
science classroom have taught us just how complex such interactions are and how little weight can be 
put on decontextualised responses of this kind. 

That the responses are used to measure levels of literacy in contemporary society and to provide 
comparisons between countries seems eminently problematic. 

That there may be other 'legitimate' answers to the questions set is a commonplace of sociolinguistic 
study of tests. [A further test item example was provided. Ed] 

How committed adults approached in their own homes by the researchers are likely to be to this testing 
discourse remains to be investigated. How are adults in their own homes engaged in these alien tasks 
expected to access that particular genre, even if they did learn it at school? It is these differences in 
discourse style and expectations about tasks, as well as the deeper assumptions about what is 
knowledge and how it is constructed and represented, that underlie what qualitative researchers refer 
to as 'cultural bias' - not simply the statistical validity of the number of tasks set. 

Issues of power 

These alternative readings are partly matters of differences between the researchers and the 
respondents, but they are also indicative of differences between research communities and their 
respective methodologies. Whatever their backgrounds, one might legitimately expect the IALS team 
with their high profile and the large consequences of their research, to take some note of the work of 
fellow researchers that impinges on their design and procedure. 

But there is a broader issue at stake here than simply academic differences. The issue here is of power: 
the interpretations placed on texts by the researchers are constantly referred to as 'correct' and where 
respondents offer different interpretations these are used to mark them down in the scales. 

If we instead pose the question of whose interpretations are 'appropriate,' applying a key concept from 
the Ethnography of Communication tradition, the frame shifts from a monolithic quasi-scientific 
assumption of truth to a more socially relative recognition that particular discourses and utterances are 
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appropriate to particular social conditions: Scribner and Cole's "specific practices promote specific 
skills." 

If the IALS researchers wish to argue that the conditions set by their survey, and in particular by the test 
items and the questions respondents were asked about them, lead to certain kinds of response as more 
'appropriate,' then they need to characterize that domain more carefully. They need to problematize its 
assumptions rather than to take them for granted and to indicate something of how other responses 
would be appropriate in other conditions. 

There is a power relation, then, between the researchers and their respondents, on the one hand, and 
between this particular style of research and other research traditions, on the other. The research team 
indeed have immense power as the very debate now going on about their findings indicates. 

That they do not draw attention to this power but instead write as though their findings are the neutral 
product of objective scientific inquiry is itself a classic procedure of institutional power. If nothing else, 
the report will provide excellent data for students interested in the workings of discursive power in late 
twentieth century Europe and N. America. 


BOX 1: "Literacy practices at work"- IALS & other research 

The report does refer to 'literacy practices at work... and in the community' and claims that 'both the range of 
practices and the relation of practice to literacy were examined'. This information was elicited through a series of 
questions to respondents 'about the frequency with which they engaged in a variety of reading, writing and 
numeracy tasks.' 

Interestingly, in the light of contemporary methodologies in qualitative research on such literacy practices, the 
researchers appear to have already drawn up a list of categories and simply asked the respondents 'how frequently 
they read or used information from six types of text', including letters or memos, reports, manuals etc. 

We are not told anything about the actual conditions in which these questions were asked, about the relations of 
questioner and respondent or about the actual discourse in which the interaction was conducted. The categories 
themselves are already closed down rather than elicited from the informants. 

We are unlikely, therefore, to learn about other kinds of indigenous literacy practices, or about their meaning to 
participants such as those outlined in a number of recent studies from around the world. In one South African 
example, a researcher observed and discussed the uses by a rural worker without formal literacy, of diagrams, head 
maps and oral interactions as he built farm wagons. The implications of this indigenous practice for educational 
programs are considerable - the need for education provision to build on local knowledge rather than marginalize 
it; the need for formal systems to articulate with informal ones; the need for preprogram research that identifies 
indigenous literacy and knowledge. The IALS report does indeed begin to recognize this issue by including questions 
about everyday literacy practices, but in comparison with these rich ethnographic accounts, its methodology 
considerably limits its value. 

While the term 'literacy practices' is frequently employed, there is equally often a slide towards more narrow 
functionally defined evaluative terms ; skills, activities, levels, tasks, and abilities are used as though they all meant 
the same as, and were a gloss on, 'literacy practices. ' 
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Box 2: A sample test item 

For example, one of the IALS test items is a passage on the 'Impatiens' plant: 

"Appearance: It is a herbaceous bushy plant with a height of 30 to 40 cm. The thick fleshy stems are branched and very 
juicy, which means because of the tropical origin, that the plant is sensitive to cold. The light green or white speckled 
leaves are pointed, elliptical and slightly indented on the edges. The smooth leaf surfaces and the stems indicate a 
great need of water" (p.32). 

The authors indicate what the item was intended to elicit: "One task asks the reader to identify 'what the smooth leaf 
and stem suggest about the plant'. 

The second paragraph of the article is labeled 'Appearance' and contains a sentence that states stems are branched 
and very juicy, which means because of the tropical origin, that the plant is sensitive to cold. This sentence distracted 
some readers from the last sentence in the paragraph: The smooth leaf surfaces and the stems indicate a great need of 
water. This task received a difficulty value of 254, placing it in the middle of Level 2" (p.32). 

The authors unusually give us the actual words used for asking the question about this complex passage, including the 
ambiguous request to identify 'what the smooth leaf and stem suggest about the plant'. The 'correct' answer is that 
these features of the leaf suggest/indicate a great need of water. How respondents took the term 'suggest' is not 
indicated nor how it was translated in the different languages used for the tests. 
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Ending the myth of the 'Literacy Myth' 

A response to critiques of the International Adult Literacy Survey (IALS) by Stan Jones, Center for the 
Study of Adult Literacy, Carleton University 

Harvey Graff (1996) and Brian Street (1996) have both claimed in recent issues of this newsletter that 
the International Adult Literacy Survey (IALS) misrepresents what literacy is and makes false claims 
about the relationship between literacy and other characteristics of individuals and societies. 

As the author of the data analysis chapters in the IALS international report, Literacy Economy and 
Society, and in the Canadian report, Reading the Future: A Portrait of Literacy in Canada, I take grave 
exception to these charges. 

Graff and Street represent a view of literacy and a view of learning and social science research in 
general, that has had a brief prominence, but has failed to deliver insights which are helpful and which 
move policy forward. I want to thank Literacy Across the Curriculum for the opportunity to address 
their misunderstandings and to show why the views that motivate the concept of literacy in IALS 
provide a credible basis for policy and action. 

The report to which Graff and Street were responding is the first of a series of reports on the survey 
and covered data collected from a small number of countries in 1994. Some of the criticisms they offer 
relate to the lack of information in that report. A technical report is now complete and is being 
prepared for publication; a preliminary version was made available to the participating countries last 
September and to those who plan to participate in a further round of data collection (and would have 
been made available to Graff and Street had they asked). Other reports on more specific topics are 
expected this year. Some of the questions the critics raise are answered in these reports. I will refer to 
them where appropriate. 

Literacy and economic success 

One of the strongest claims in IALS is that there is an important relationship between literacy skill and 
individual economic success. 

The evidence for this lies in a number of observations in the study: 

• There is a relationship between income and literacy skill. For example, in Canada IALS found that over 
80% of those at the lowest literacy level had incomes below the median quintile, but just 42% of those 
at the highest level had incomes this low. Over a quarter of those at the highest level had incomes in 
the top quintile, but only 5% of those at the lowest level had incomes this high (p. 61 of the report) 

• There is a relationship between labour force attachment and literacy. For example, in Germany the 
average score of those working in industries showing the greatest growth was significantly higher than 
the average score of those working in essentially stagnant industries (p. 65). 
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Problems with historical perspectives 

Against this, Street and Graff refer to Graff's 1979 study of literacy in mid-19th century Ontario where 
he found large numbers of individuals with literacy skill who had not experienced economic success. 
There are a number of reasons why this Study, however interesting historically, is largely irrelevant to 
current policy. 

I have never understood why researchers such as Graff and Street who argue that literacy is narrowly 
specific to time and place should assume that relationships between literacy and anything that held 
over 100 years ago should necessarily hold today. Surely, any sensible understanding of how societies 
change must allow for changes in the relationship between personal characteristics and life chances. 
Sennett and Cobb (1972), in one of the most insightful books on what ability is and how it functions in 
society, argue persuasively that ability, (of which literacy is but one example) only began to have a 
significant role in occupational opportunity after (and as a consequence) of the French Revolution. 

Thus, I know of no logical bar to Graff being correct about 19th century Ontario and IALS being correct 
about late 20th century Ontario and Canada, the United States and the other IALS countries. Bulcock 
and Wang (1993). using data from a previous Canadian survey of adult literacy - the Survey of Literacy 
Skills Used in Daily Activities - also show that Graff is wrong about 1990 Canada, except for those 
immigrants who speak neither official language. 

It seems to me that even for the 19th century, Graff only convincingly shows that half the connection is 
missing. By showing that there were large numbers of individuals who had literacy skills but not 
economic success, he has shown only that literacy is not a sufficient condition for such success. To 
demonstrate that it is not a necessary condition, he would also have to show that as a general 
condition large numbers of those who have experienced economic success do not have literacy skills. 

The presentation of data on this is quite clear in IALS; there are tables that show the incomes of those 
at different literacy skills and tables that show the literacy skills of those at different incomes. 

This issue is explored further in the Canadian report where it is shown that the relationship between 
income and literacy is stronger in the United States and Canada than it is in Germany and the 
Netherlands. The labour market in the European countries is generally considered to be less flexible 
than that in the North American ones.i Under most theories of the labour market, we would expect to 
find a greater connection in flexible than in rigid markets, just the result that IALS found. The study 
makes no claim that the stronger relationship in the North American economies is more desirable. It 
must be pointed out, however, that many labour economists attribute some of the relatively high 
unemployment now found in European economies to such rigidity. 

Measures of literacy 

One of the problems facing Graff in his study, was that he had to work with relatively crude measures 
of literacy, ones that were mostly 'have/don't have it' measures. 

One of the advances of IALS, an advance acknowledged by these critics, is that it provides a more 
sophisticated measure of literacy skill. One of the strongest findings of IALS is that simply 'having it' is 
not a very useful measure. 
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Individuals who are 'literate' by any of the conventional 'literate/illiterate' definitions may nonetheless 
be relatively low skilled and, as such, have less access to good jobs and the high wages they offer. 

This issue of relative skill differences is further explored in my Reading, but not Reading Weii (Jones, 
1993). Had Graff had a more nuanced measure of literacy to work with, he might not have been able to 
'disprove the literacy myth.' 

Skills and jobs 

Graff claims that economists see little connection between skill and job attainment and performance. 
While it is undoubtedly true that one can find an economist to support almost any position, this is 
clearly an overstatement. The most reputable research group on labour market issues, the National 
Bureau of Economic Research (NBER), has consistently shown in its work that skill (represented by 
education and training) is indeed important in job attainment and performance. 

Much of the evidence Graff and Street cite for the lack of connection between skills and jobs comes 
from qualitative studies of one or more workplaces with a small number of subjects. What they don't 
say is that there is an equal number of qualitative studies that show skill is important to job 
performance and attainment. A careful, though now dated, review by Spenner (1985) concluded that 
neither position had been strongly supported, Thus new data, such as that from IALS, can serve to 
clarify the situation. 

As Bailey (in press), one of the more comprehensive analysts of changes in skill and work requirements, 
points out, studies of this relationship have been plagued by difficulties in measuring skill. 

Because IALS offers a high quality tool for measuring skill, it can help improve these studies. For 
example, many past studies have used educational attainment as a substitute for skill, but anyone who 
has been near a secondary school in this century knows that high school graduation represents a very 
wide range of skills. The power of a cognitive assessment such as that used in IALS is that it offers a 
more precise measure of skill. Hence we would expect to find stronger relationships between skill and 
work force characteristics using IALS than when using the education proxy, and we do. 

In all, a fair reader of the changing work force literature would conclude that the evidence from the 
1990s increasingly, shows that good jobs require higher skills and that there are good jobs being 
created. The most extensive study of this issue, the OECD Jobs Study, certainly came to that conclusion 
(OECD, 1994). 2 

Graff and Street are correct that there are lots of jobs with low skill requirements being created. What 
they appear to believe is that there are no jobs with high skill requirements being created. They are 
wrong. They appear to believe that high skills are not required for good jobs. They are wrong and, 
worse, their advice is dangerous because it could lead policy makers to believe that investments in 
literacy are not important. It is interesting that neither Graff nor Street cite any labour market 
economists in support of their view. The only citations in Street's bibliography that refer to the 
changing workplace are to works written by linguists and discourse theorists. 

The analysis of the economic connections in the international report is necessarily brief and, as such, 
only touches generally on these issues. Stephen Raudenbush and his colleagues (Fotiu, Raudenbush, et 
al,1995) are engaged in a more substantial analysis of the matters with the IALS data. To date their 
analysis has confirmed and extended the broad outlines presented in the international report. It should 
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also be noted that Andrew Sum, in an as yet unreleased paper on the National Adult Literacy Survey in 
the United States, argues that the connection appears weaker when all ages are considered than when 
only experienced workers are analyzed. 

In North America, employers have little information about the skills of the young adults they hire (and 
seem to make scant use of the little they do have) so their hiring decisions are more likely to be based 
on credentials rather than skill. Once employees have been hired, however, the employer can observe, 
judge their skills, and reward them. Thus we would expect to find a weaker relationship when young 
workers are included, as they were in the IALS report. Sum also shows that once the differences in 
income that are due to differences in literacy are taken into consideration, the differences usually 
attributed to ethnicity, and gender disappear or are minimized, again providing evidence that at some 
point skill is being directly recorded independently of other factors. In short, one of the enduring myths 
of academic literacy research is that Graff laid to rest 'the literacy myth' once and for all. He did not. 
Data from IALS and from other labour market research demonstrate that whatever the relationship 
between literacy and economic success over 100 years ago, the relationship in the 1990s is real and 
important. 

I stress that this is an academic belief because I have met few literacy practitioners and even fewer 
literacy learners who believe that the 'literacy myth' has been refuted; their daily lives provide 
sufficient counterevidence to Graff's claim. 


Literacy and cognitive outcomes 

Street offers another criticism, claiming that Scribner and Cole (1981) have demonstrated that literacy 
does not have cognitive consequences and that this finding invalidates the findings in IALS, although 
IALS makes no direct claims about the cognitive outcomes of differences in literacy skill. 

This is a stronger claim about their work than Scribner and Cole themselves make. In any case, Keith 
Stanovich (1993) has show that an alternative interpretation of Scribner and Cole's results is just as 
plausible. This interpretation is more consistent with the substantial body of evidence that literacy, does 
have cognitive outcomes. Scribner and Cole studied the difference between individuals among the Vai, a 
society in Liberia, that created their own native language literacy. They found that individuals who were 
literate only in this Vai script did not show significant general cognitive differences from those who were 
not literate in any language, but that those who were literate in English - the language of education - 
did. Since one difference between Vai and English literacy is that the latter but not the former is taught 
in school, it might be argued that the cognitive consequences associated with English literacy but not 
with Vai are a result of schooling rather than of literacy itself. This, as I understand Street, is the claim he 
wants readers to accept. 

Stanovich (1993), however, points out several other important differences between Vai and English 
literacy 

• Vai literacy is not a general literacy, like English literacy, but a very restricted one, used only for 
personal letter writing and business communication between individuals who are familiar with 
each other, and thus not used for anonymous communication. 

• In particular, Vai literacy does not include the essayist style that Olson (one of the leading 
proponents of cognitive consequences) argued was central to literacy's effect. 
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• Vai literacy is learned in the late teens or twenties, not as a child. 

• There are no libraries or books in the Vai script and so Vai does not provide access to general 
world knowledge. 


Notably, Scribner and Cole themselves call attention to these limitations of their own study, pointing out 
that "Vai script literacy does not fulfill the expectations of those social scientists who consider literacy a 
prince motor in social change" (p. 239). Enthusiasts such as Street seldom bother with the original 
authors' own qualifications of their findings and so grossly overstate the impact of those findings 
regarding the consequences of literacy. Since there is considerable literature, of the same high quality as 
Scribner and Cole's work, but without the limitations identified here, that does demonstrate cognitive 
consequences, Street's criticism has little relevance for assessing the quality, or acceptability, of IALS. 

Purposes of literacy: Two views 

Both Street and Graff claim that ethnographic research has shown that literacy can be used for different 
purposes and that, for some individuals at least, literacy is used for only some of those purposes. From 
this, they wish to conclude that there not only is not, but that there can not be any general literacy skill. 

None of the research they cite has to my knowledge shown that the cognitive processes involved in 
literacy are different when used for different purposes. 

Typical of such studies is Gee (1988) who speculates about the knowledge and attitudes necessary to 
'read' an aspirin bottle; he never presents any empirical evidence that individual differences in the 
ability to do so are a function of the presumed differences in knowledge and attitude and not of 
differences in some general reading ability. The dispute between advocates of situated cognition 
(Street's and Graff's beliefs fall squarely into this camp) and those who work in the cognitive science 
tradition (a tradition that I believe IALS sits squarely in the middle of) have occupied research journals in 
education for nearly a decade. Despite what Street and Graff would have us believe, the issue is far from 
settled. 

This is hardly the place for an extended discussion of this dispute.3 It is important to note that Greeno, a 
leading theoretician of the situated learning school, accepts several positions that would seem to be 
inconsistent with the claims Street sets out. These are that: 

• Learning need not be bound to the specific situation of its application. This is fully consistent 
with the large body of work in cognitive psychology which demonstrates the existence of 
transfer of learning. 

• Knowledge can indeed transfer between different sorts of tasks. 

• Abstract instruction can be very effective and one need not teach everything in concrete, almost 
vocational settings. 

• Instruction need not take place only in complex social situations. (Anderson, Reder, and Simon, 
1997 
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In short, cognitive skills are not exclusively situation-specific skills, and a broad-based approach to 
assessing these skills is appropriate. What we can expect is that the more secure the skills, the more 
successfully they will be transferred. Indeed, one way of thinking about the literacy scales in studies 
such as IALS is that high scores represent the most transferable level of skill while each succeeding lower 
level indicates greater and greater difficulty in applying the skills in new situations. I have argued (in 
Reading, but not Reading Well) that the lowest level of skill may indeed represent a non-transferable 
level of skill. 

The specific evidence from the statistical analysis of IALS is that there is simply no general evidence of a 
situation effect. If previous experience were all that counted, as Street seems to say, then we would 
expect someone who answered one of the questions based on the bus schedule correctly to answer the 
rest of the questions on the bus schedule correctly. This doesn't happen. Instead, the best predictor of 
whether someone can answer a particular question on the bus schedule correctly is how well they 
answer questions with similar cognitive demands on other, situationally unrelated texts. 

But let's assume that Street and Graff are right that literacy is narrowly situationally specific. Then it 
would seem that IALS has managed to locate and test a situationally specific literacy that is highly 
related to social and economic well-being. 

Street and Graff are sure to claim that we have thus privileged this one kind of literacy. But it is not the 
IALS researchers who have privileged it, it is society. While we might determine test scores, we don't 
determine employment, income, social participation or any of the other characteristics we found 
associated with IALS literacy. It is not for the IALS research team to determine whether it is fair that this 
one kind of literacy is so valued by society. It would have been negligent of us, however, having 
discovered these connections not to have reported them. 

Interpreting IALS task responses 

Street further claims that we narrowly construed the answers we accepted, penalizing those who might 
bring new interpretations and insights to particular questions. 

The scorers probably wish that we had done so, because that would have made their tasks simpler. 
Instead, we reviewed novel answers and continually added new correct responses to the scoring 
protocol. All of the IALS tasks were in constructed response format (we used no multiple-choice 
questions), so we did get answers that we had not anticipated. Some of these were as good as ones we 
did anticipate and we accepted them and had our scorers include them in their protocols; others were 
as off the mark as wrong answers we anticipated and we did not accept them. We did not ask our 
scorers to act as Scantron machines and mark right and wrong mechanically. The sample of tasks 
published in the report is less representative than might be wished and this has allowed Street to 
criticize the narrowness of the test. [See BOX 31 Our publishing choices, however, were limited. We 
could not publish all the tasks because the assessment was then in use in four other countries and we 
anticipated that it would be used with a third group as well. To have released the test as a whole would 
have jeopardized the results in those studies. We thought that not publishing any examples would also 
be a disservice, as it might then be difficult for readers to understand how the test tasks were 
constructed. For the moment (actually for quite a long time as the schedule for use of the test now 
extends into the next century), readers will simply have to accept that we used a broad range of items 
from a great variety of contexts, topics, situations and difficulties. 
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Although no more tasks will be published in the technical report soon to be released, a more complete 
description of the task pool will be available there. Survey respondents who scored high did so because 
they had skills that allowed them to deal with tasks across that range represented on the test: on 
different topics, from different contexts, in different situations, and at different levels of difficulty. Those 
who scored lower had skills that were more restricted. What else could differences in skill mean? 

Literacy and equity 

What I agree on with Street and Graff is that access to high literacy skills has not been, and is not now, 
equitably distributed in society, I do not see, however, what this has to do with whether or not 
differences in literacy skill matter. Graff seems to believe that because differences in literacy are more 
closely tied to differences in social class and opportunity than to economic success, this disproves any 
connection between literacy and economic success. 

I have no trouble understanding that parents' social economic status has an influence on the 
opportunity to acquire skills and that skills thus acquired or not acquired have an effect on economic 
well-being. Path analyses of this kind are now the stock in trade of social science research. 

It is important to understand, however, that the connection between parental circumstances and child's 
literacy as an adult is an empirical issue. Doug Willms, working with the IALS Canadian data, presents 
strong evidence in a paper soon to be released in the IALS monograph series that the connection varies 
considerably from province to province with some showing little or no effect and others showing 
marked effect. In all, however, there is a connection between literacy, and involvement in the economy. 

The criticisms of Graff and Street are not new. They are, however, inaccurate. We ask readers to judge 
IALS on what it says about the importance of literacy to all adults. Literacy ability is a factor in work force 
participation, it is a factor in social participation. IALS shows this to be the case. 

In many ways the IALS message is not new to the literacy field. Anyone who has worked with learners 
understands that this message is not new to them. It does appear to be new to the people who decide 
whether there should be funding for literacy programs. The message from the literacy research 
community in Canada that literacy didn't matter, the same message Street and Graff want us to believe, 
had nearly succeeded in convincing governments that investments in adult literacy were not 
worthwhile. 

The message from the literacy practitioner community, especially those in the work force literacy and in 
the basic skills training communities, had been different. We like to think that the support for that 
message in IALS helped convince those who determine government spending that investments in 
literacy are worthwhile. It may be just hubris on our part, but those of us who worked on IALS in Canada 
do draw a connection between the release of our Canadian report in September 1996 and the increased 
funding for the National Literacy Secretariat, especially for workplace literacy, in the federal budget in 
February 1997. As well, literacy workers in several provinces have told us that the international report 
was instrumental in preserving or increasing funding for literacy. Hopefully, IALS has ended another 
'literacy myth,' the myth that literacy doesn't matter. 
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Notes 

1. See Freeman (1994) for an accessible discussion of comparative labour market rigidity and its 
consequences. 

2. Those who want a comprehensive review of the literature on skills and jobs will find Harry 
O'Neil's Workforce readiness: Competencies and assessment (in press) an important source. 

3. Readers interested in pursuing the matter will find that the exchange in Anderson, Reder and 
Simon (1996), Greeno (1997), and Anderson, Reder and Simon (1997) provides the most recent 
discussion of the two positions. 


Box 3: The challenge of analyzing tasks across cultures 

I cannot blame Street for this misinterpretation, as we were less clear in our explanations of this than we could 
have been. Similarly, our analysis procedures allowed a task in one country to have different difficulty levels than 
in others. This happened in only a few cases and in almost all of them it was because we had not been as diligent in 
ensuring format congruence from language to language as we should have been. For example, one task based on a 
letter to the editor about cloth versus disposable diapers was considerably more difficult in the Netherlands than 
in other countries. 

This was somewhat surprising as the letter had originally appeared in a Dutch newspaper and contained 
information that might be considered 'Netherlands specific'; under the situational hypothesis this should have 
made the task easier for Dutch respondents, not more difficult. It turns out that in translating the text the original 
Dutch word 'five' had been changed to the numeral '5 1 . 

Since '5' stands out in a text more easily than 'five 1 , those taking the test in other languages found it easier to 
locate this crucial piece of information. This is consistent with our cognitive explanation of performance on the 
test. Further evidence from an analysis of other country-to-country differences shows that almost all can be 
accounted for by such textual rather than situational differences. 


The Centre for Literacy 


21 





